The present invention generally relates to the field of video image processing and, more particularly, to techniques for filtering noise from digital video pictures.
The present invention is useful for filtering digital video sequences corrupted by high noise levels. Because of the particular importance of the Moving Pictures Experts Group (MPEG) standard in treating digitized video sequences, reference will be made to an MPEG2 system to illustrate an implementation of the present invention. Of course, those of skill in the art will appreciate that the invention may also be used in various systems for transferring video sequences based on different standards, as established from time to time. The main application of the invention is picture pre-processing before the MPEG2 or other standard coding. Even so, it is possible to use the invention outside a coding process, such as in a TV set for filtering the pictures to be displayed upon reception, for example.
Pre-processing pictures to be coded according to the MPEG2 standard provides great enhancement of the coding efficiency. For example, see A. van der Werf et al., xe2x80x9cI.McIC: a single-chip MPEG-2 video encoder for storagexe2x80x9d, IEEE Journal of Solid-State Circuits, Vol.32, n. 11, November 1997; and L. Yan, xe2x80x9cNoise reduction for MPEG type of codexe2x80x9d, Proc. IEEE Int. Conf Acoust., Speech, Signal Process., 1994. Additionally, many filtering techniques of pictures have been developed. See J. C. Brailean, R. P. Kleihorst, S. Efstratiadis, A. K. Katsaggelos and R. L. Lagendijk, xe2x80x9cNoise reduction filters for dynamic image sequences: a review,xe2x80x9d Proc. IEEE, vol. 83, pp. 1270-1292, September 1996. In many cases, burdensome aspects of these techniques may have include high calculation power requirements or the need for several iterations on the same picture. As a consequence, these methods are either too expensive or too difficult to implement for real-time video applications.
Prior art real-time video filtering techniques (see, e.g., E.Dubois and S.Sabri, xe2x80x9cNoise reduction in image sequences using motion-compensated temporal filteringxe2x80x9d, IEEE Trans. on Communications, vol. COM-32, pp. 826-831, July 1984) typically base the filtering operations on the separation distances among the unfiltered grey level of the pixel to be filtered (Po) and the grey levels of other pixels of a working window. The effectiveness of the filtering in this case is inversely proportional to the size of these distances. As a consequence, the pixels corresponding to such large distances are not involved in the filtering operations. This is a coarse method of segmenting the picture that avoids the calculation of the average of pixels belonging to different objects. This operation would cause an out of focus of the picture and, in the case of a temporal filter, also the appearance of xe2x80x9ctails.xe2x80x9d
The document WO 97/30545 discloses a motion-compensated recursive filter based on the above method. Such a filter, depending on the absolute value of the difference between the current pixel and the corresponding pixel in the preceding picture, establishes the value of a coefficient or fraction xcex2. The filter further combines the fraction xcex2 of the current pixel with a fraction 1-xcex2 of the corresponding pixel in the preceding motion-compensated picture. Such a system is substantially an infinite impulse response (IIR) adaptive filter of the first order. This technique has the drawback of leaving unchanged the current pixel if it is too different from the corresponding pixel of the preceding picture, even if such difference could be due to noise.
A spatial adaptive low pass filter whose adapting mechanism is based on the calculation of the difference among Po and the other pixels of the working window is disclosed in EP 878,776 A1. A local evaluation of certain parameters of the picture is carried out (for example, whether the considered pixel belongs to a uniform zone or not). On that basis it may be determined whether the pixel to be filtered is near to or far from the other pixels, by way of a fuzzy logic process. Near pixels are given more weight than far pixels in the calculation of the value of the filtered pixel. Yet, even in this case the effectiveness of such a filter in the presence of high noise levels is not high. This is because the decision that is made to assess whether the considered pixel belongs to a uniform zone (and therefore if it is possible to filter more) or not is based on distances among P and the surrounding pixels.
In the article entitled xe2x80x9cA method of noise reduction on image processingxe2x80x9d, IEEE Transactions on Consumer Electronics, Vol. 39, N. 4, November 1993, by S. Inamori et al., a temporal filter is described that is based on the above-described filter of Dubois and Sabri but which is more sophisticated. That is, the input-output characteristic is adaptively chosen as a function of the expected power of the noise. The filter is turned off by signals coming from a motion detection section to avoid the generation of trails and by an edge detection block to prevent causing the picture to be out of focus. Even so, this filter suffers similar drawbacks to the previously mentioned prior art filters and must be turned off in the presence of noise peaks.
These known techniques become unsatisfactory in the presence of high noise levels. It becomes necessary to turn off the filter every time a noise peak (xe2x80x9cspikexe2x80x9d) is superimposed on the pixel Po being processed. High noise peaks are noise samples of a value much higher than other samples. Although these noise peaks have a rather small probability because the queues of the stochastic distribution of many kinds of noise (e.g. Gaussian noise) are theoretically infinite, they occur nonetheless. In such cases, the above mentioned prior art filters always interpret higher than normal differences among the current pixel and the neighboring pixels as if the picture is not stationary and not as if the picture is corrupted by noise. Therefore, the performance of noise reduction of the filters is significantly limited.
A well known technique for selecting the pixels of the working window is the so called xe2x80x9cDuncan Range Testxe2x80x9d (DRT) (see D. B. Duncan, xe2x80x9cMultiple range and multiple f-testsxe2x80x9d, Biometrics, vol. 11, pp. 1-42, 1955) which has been used with good results. For example, see R. P. Kleihorst, xe2x80x9cNoise filtering of image sequences,xe2x80x9d Ph.D. Thesis TU-Delft, Information Theory Group, 1994. In order to highlight the characteristics of the DRT, a brief description follows. Suppose a set of data, e.g., the luminance values of the pixels to be filtered, is organized in an increasing order as follows:
g(l)xe2x89xa6g(2)xe2x89xa6 . . . xe2x89xa6g(n)
where the number in parentheses indicates the position assumed in the ordered set. The DRT is based on the definition of a set of xe2x80x9csimilarxe2x80x9d values, meaning that two values are considered xe2x80x9csimilarxe2x80x9d if:             "LeftBracketingBar"                        g                      (            i            )                          -                  g                      (            j            )                              "RightBracketingBar"              σ      n        ≤      ρ                  "LeftBracketingBar"                  i          -          j                "RightBracketingBar"            ,      a      
where "sgr"n is the standard deviation of the superimposed noise that is considered known. xcfx81|i-j|,a is a value that depends on the precision level a of the test, by the number |i-j| of data, that in the specific case is a number of pixels between g(i) and g(j), and by the stochastic distribution of the noise. xcex1 represents the probability of taking a wrong decision. Obviously, the smaller xcex1 is, the smaller xcfx81 is, and so the more restrictive the test is.
With these premises, the object of the DRT is to find the sub-set including the current pixel Po and including the greatest number of xe2x80x9csimilarxe2x80x9d pixels. In other words the following sub-set of pixels must be determined:
g(i), . . . , Po, . . . , g(j)
where i and j are such that:             j      -      i        =                  MAX                              i            xe2x80x2                    ,                      j            xe2x80x2                              ⁢              xe2x80x83            ⁢              (                              j            xe2x80x2                    -                      i            xe2x80x2                          )            ⁢              xe2x80x83            ⁢      where      ⁢              xe2x80x83            ⁢              j        xe2x80x2            ⁢              xe2x80x83            ⁢      and      ⁢              xe2x80x83            ⁢              i        xe2x80x2            ⁢              xe2x80x83            ⁢      satisfy      ⁢              xe2x80x83            ⁢      the      ⁢              xe2x80x83            ⁢      conditions      ⁢              :                  {                                                                      "LeftBracketingBar"                                                      g                                          (                      i                      )                                                        -                                      g                                          (                      j                      )                                                                      "RightBracketingBar"                                            σ                n                                      ≤                          ρ                                                "LeftBracketingBar"                                      i                    -                    j                                    "RightBracketingBar"                                ,                a                                                                                                    g                              (                                  i                  xe2x80x2                                )                                      ≤                          P              o                        ≤                          g                              (                                  j                  xe2x80x2                                )                                                        
The successive filtering operations are performed only on the pixels belonging to the selected range. Notably, the DRT does not have as a unique reference value the value of Po in establishing which pixels are to be filtered together with the current pixel and which pixels are not to be filtered. This test is substantially based on an external measurement of the standard deviation of the noise "sgr"n to test all intervals containing Po, not necessarily those symmetrical to it, and select the interval containing the largest number of data (i.e., of pixels).
The DRT has been implemented in filters for digital pictures. Such filters, showing a relative insensibility to the value to be filtered Po, allow more effective filtering in rather uniform zones where the noise is more readily detectable, though safeguarding the contours of objects and avoiding the formation of xe2x80x9ctrails.xe2x80x9d Yet, despite the relative simplicity with which the DRT may be described, it may be difficult to determine the set of data to process. In fact, the amplitude of the interval, given by the product "sgr"nxc2x7xcfx81|ixe2x80x2-jxe2x80x2|,a, depends on many factors, such as the precision level xcex1 of the test, the noise level "sgr"n, and the number of pixels that are included in this range. The latter dependency, in particular, greatly complicates the algorithm. This is because it makes necessary the testing of a great number of sub-sets of pixels containing Pbefore being able to identify the sub-set containing the largest number of data.
Accordingly, there is a need for a filter for digital video pictures that, while retaining the advantages of the DRT, does not leave unattenuated large noise peaks. This would severely degrade the efficiency of the MPEG2 coding and of any other system of video compression downstream of the filter. Furthermore, such filter should not involve a high computational complexity. Otherwise, the cost for real-time video applications would be prohibitive.
It is an object of the present invention to provide a new filtering method and filter for digital video pictures having the advantages of the DRT while substantially eliminating the noise peaks and being relatively easy to implement, making it relatively inexpensive for use in real-time video applications, for example.
According to the present invention, a method of filtering noise from digital pictures includes selecting a first set of pixels including a pixel to be filtered from the current picture and a second set of pixels temporally and spatially near the pixels of the first set. The method may also include calculating a certain number of extended sums of values assumed by as many pre-established weight functions of the intensities of a filtered selected video component on the first group of pixels. Furthermore, the intensity of the filtered selected video component of the pixel to be filtered may be calculated as a function of the extended sums and of the sums weighted with the weight functions of the intensities of the selected video component over the first set of pixels.
The method of the present invention may be implemented even while choosing the pixels of the second set of pixels exclusively from the current picture or from a preceding picture. Each of the pre-established weight functions may be made specular to a respective central value calculated as a function of the unfiltered selected video component in the pixel to be filtered and of the noise level. Furthermore, the intensity of the selected filtered video component of the pixel to be filtered may be a weighed average with pre-established weights of the ratios among the weighed sums and the extended sums.
A noise filter for digital pictures according to the present invention includes a certain number of first circuit blocks, each being input with a set of pixels including a pixel to be filtered from the current picture and a second set of pixels temporally and spatially near the pixel to be filtered. Each block may produce an extended sum of the values assumed by a respective pre-established weight function of intensity of a selected video component on the first set of pixels. Furthermore, each block may produce the weighted sum with the respective weight function of the intensities of the selected video component on the first set of pixels. A second circuit block, having inputs coupled to the outputs of the first circuit blocks, may calculate the intensity of the filtered selected video component in the pixel to be filtered as a function of the weighed sums and the extended sums.
Each weight function may be specular to a respective central value calculated as a function of the intensity of the unfiltered selected video component of the pixel to be filtered and of the noise level. Also, the weight functions of the filter may be determined using tail detection.